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This paper studies the relationship between disjunctive form, a syntactic normal form for the modal 
ji calculus, and the alternation hierarchy. First it shows that all disjunctive formulas which have 
equivalent tableau have the same syntactic alternation depth. However, tableau equivalence only 
preserves alternation depth for the disjunctive fragment; there are disjunctive formulas with arbi¬ 
trarily high alternation depth that are tableau equivalent to alternation-free non-disjunctive formulas. 
Conversely, there are non-disjunctive formulas of arbitrarily high alternation depth that are tableau 
equivalent to disjunctive formulas without alternations. This answers negatively the so far open ques¬ 
tion of whether disjunctive form preserves alternation depth. The classes of formulas studied here 
illustrate a previously undocumented type of avoidable syntactic complexity which may contribute 
to our understanding of why deciding the alternation hierarchy is still an open problem. 


1 Introduction 

The modal jj. calculus Q, L^, is a modal logic augmented with its namesake least fixpoint operator /r 
and the dual greatest fixpoint operator, v. Alternating between these two operators gives the logic its 
great expressivity fTj while both model checking and satisfiability remain pleasingly decidable. The 
complexity of model checking is, at least currently, tied to the number of such alternations, called the 
alternation depth of the formula being checked fTOll . The problem of deciding the least number of 
alternations required to express a property, also known as the Rabin-Mostowski index problem, is a long 
standing open problem. 

Disjunctive normal form is a syntactic restriction on formulas which first appeared in f91| and was 
then used as a tool for proving completeness of Kozen’s axiomatization llT4l . It is based on the tableau 
decomposition of a formula which forces it to be in many ways well-behaved, making it a useful tool 
for various manipulations. For instance, satisfiability and synthesis are straight-forward for disjunctive 
formulas. In l21 it is used to analyse modal from a logician’s perspective. More recently, disjunctive 
form was found to allow for simple formula optimisation: if a formula is equivalent to a formula with¬ 
out greatest fixpoints, then such a formula is easily produced by simple syntactic manipulation on the 
disjunctive form of the formula ifTTI . 

Each of these results uses the fact that any formula can be effectively transformed into an equivalent 
disjunctive formula with the same tableau - indeed, disjunctive form is perhaps the closest one gets to 
a canonical normal form for L^. The transformation itself, described in ||9l, is involved and it has so 
far been an open question whether it preserves the alternation depth of formulas. If this was the case, it 
would be sufficient to study the long-standing open problem of the decidability of the alternation hierar¬ 
chy on this well-behaved fragment. 

In this paper, we show that although the disjunctive fragment of is itself well-behaved with respect 
to the alternation hierarchy, the transformation into it does not preserve alternation depth. 
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The transformation into disjunctive form takes the tableau decomposition of a formula, and produces 
a disjunctive formula that generates the same tableau. The first contribution of this paper is to show 
that all disjunctive formulas generating the same tableau have the same alternation depth. This result 
brings some clarity to the transformation into disjunctive form since one of the more difficult steps of the 
construction is representing the parity of infinite paths of the tableau with a finite priority assignment. 
The result presented here means that all valid choices are just as good, as all yield a disjunctive formula of 
the same alternation depth. As a result, the alternation hierarchy is decidable for the disjunctive fragment 
of with respect to tableau equivalence, a stricter notion of equivalence than semantic equivalence, as 
defined in ifldl . 

The second confribufion of fhis paper is fo show fhaf fhis does nof exfend fo non-disjuncfive formulas. 
Nof only does fableau equivalence nof preserve alfemafion depfh in general, buf fhe alfemafion depfh of a 
formula does nof guaranfee any upper bound on fhe alfemafion depfh of equivalenf disjuncfive formulas. 
Indeed, for arbifrarily large n, fhere are formulas wifh a single alfemafion which are fableau equivalenf 
only fo disjuncfive formulas wifh af leasf n alfernafions. 

Conversely, fhere are formulas of wifh arbifrarily large alfemafion depfh which are fableau equiv¬ 
alenf fo a disjuncfive formula wifhouf alfernafions. This shows fhaf fhe alfemafion depfhs of fableau 
equivalenf formulas are only direcfly relafed wifhin fhe disjuncfive fragmenf. 

The signficance of fhese resulfs in fwofold. Firsf, fhey oufline fhe limifs of whaf can be achieved 
using disjuncfive form: disjuncfive form does nof preserve alfemafion depfh so despife being a useful 
fool for safisfiabilify-relafed problems, if is unlikely fo be of much help in confexls where fhe alfemafion 
depfh of a formula mailers, such as model-checking or formula oplimisafion beyond fhe firsl levels of 
fhe alfemafion hierarchy. 

Secondly, and perhaps mosl significanlly, fhese resulfs impacl our underslanding of fhe alfemafion 
hierarchy. This paper’s resulfs imply fhaf deciding fhe alfemafion hierarchy for fhe disjuncfive fragmenf 
of L^, an open buf easier problem, is nof sufficienl for deciding fhe alfemafion hierarchy in fhe general 
case. The counferexamples used fo show fhis illuslrale a previously undocumenled lype of accidenlal 
complexify which appears fo be difficull fo identify. These may shed lighl on why deciding fhe alfemafion 
hierarchy is slill an open problem and examplify a cafegory of formulas wifh unnecessary alfernafions 
which need fo be lackled wifh novel melhods. 

Related work Deciding fhe modal fX alfemafion hierarchy is exaclly equivalenf fo deciding fhe Rabin- 
Moslowski index of alfemafing parily aufomafa. The corresponding problem has also been sfudied for 
aufomafa operafing on words Q and aufomafa which are deferminisfic |[T3l HU, or non-deferminisfic 
0 13 rafher fhan alfemafing. As will be highlighfed fhroughouf fhis paper, many of fhe melhods used 
here are similar fo melhods applied fo differenl fypes of aufomafa. 


2 Preliminaries 

2.1 The modal jx calculus 

For clarify and conciseness, fhe semanfics of are given direcfly in terms of parity games. As is well 
documented in the literature, this approach is equivalent to the standard semantics ||2l. The following 
definitions are fairly standard, although we draw the reader’s attention to the use of the less typical 
modality in the syntax of and the unusual but equivalent definition of alternation depth. 
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Definition 1. (L^) Given a set of atomic propositions Prop = {P, 2, ...} and a set of fixpoint variables 
Var = {X,F,...} , the syntax of is given by: 

0 := T I _L I P I -iP I X I 0 A 0 I 0 V 0 I where ^ is a set of formulas | pX.^ \ vX.^ 

The modality replaces the more usual modalities 00 and 00. If is a set of formulas, 
stands for 00) A □ V(|)6^ 0: every formula in SS must be realised in some successor state and each 

successor state must realise at least one of the formulas in SB. The modalities 00 and 00 are expressed 
in this syntax by —>-{0, T} and —)-{0} V —)-_L respectively, where _L denotes the empty set. 

Without loss of expressivity, this syntax only allows for formulas in positive form: negation is only 
applied to propositions. Furthermore, without loss of expressivity, but perhaps conciseness, we require 
all formulas to be guarded: all fixpoint variables are within the scope of a modality within their binding 
formula. For the sake of clarity, we restrict our study to the uni-modal case but expect the multi-modal 
case to behave broadly speaking similarly. To minimise the use of brackets, the scope of fixpoint bindings 
should be understood to extend as far as possible. 

Definition 2. (Structures) A structure .y# = {S,so,R,P) consists of a set of states S, rooted at some initial 
state So € S, and a successor relation RC S x S between the states. Every state s is associated with a set 
of propositions P{s) C Prop which it is said to satisfy. 

Definition 3. (Parity games) A parity game is a potentially infinite two-player game on a finite graph 
W = {Vo,Vi,E,vj,Q.) of which the vertices VoUFi are partitioned between the two players Even and Odd 
and annotated with positive integer priorities via O : Fq U Fi —)• N. The even player and her opponent, 
the odd player, move a token along the edges E C Vb U Vi x Vb U Fi of the graph starting from an initial 
position V/ € Vb U Fi, each choosing the next position when the token is on a vertex in their partition. 
Some positions p might have no successors in which case they are winning for the player of the parity 
of A play consists of the potentially infinite sequence of vertices visited by the token. For finite 

plays, the last visited parity decides the winner of the play. For infinite play, the parity of the highest 
priority visited infinitely often decides the winner of the game: Even wins if the highest priority visited 
infinitely often is even; otherwise Odd wins. Note that since some readers may be used to an equivalent 
definition using the lowest priority to define fhe winner, whenever possible, “mosf significanl” will be 
used fo indicate fhe highesf priority. 

Definition 4. (Strategies) A posifional sfrafegy a for one of fhe players in = (Fo,Fi,E,v/,0) is a 
mapping from fhe player’s positions s, in Fq for Even and in Fi for Odd, in fhe game fo a successor 
position F such fhaf {s,s')eE. A play respecfs a player’s sfrafegy a if fhe successor of any position in 
fhe play belonging fo fhe player is fhe one dicfafed by a. If a is Even’s sfrafegy and T is Odd’s sfrafegy 
fhen fhere is a unique play a x T respecfing bofh sfrafegies from every position. The winner of fhe parity 
game af a posifion is fhe player who has a sfrafegy a , said fo be a winning sfrafegy, such fhaf fhey 
win a X T from fhaf posifion for any counfer-sfrafegy T. The following sfafes fhaf such sfrafegies are 
sufficienf: players do nof need fo fake info accounf fhe hisfory of a play fo play opfimally. 

Fact 5. Parity games are positionally determined: for every position either Even or Odd has a winning 
strategy Ml/. 

This means that strategies gain nothing from looking at the whole play rather than just the current 
position. As a consequence, we may take a strategy to be memoryless: it is a mapping from a player’s 
positions to a successor. 
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For any formula 0 and a structure ^ we define a parity game ^ x 0, constructed in polynomial 
time, and say that ^ satisfies written ^ |= 0, if and only if the Even player has a winning strategy 
in ^ X 0. 

Definition 6. (Model-checking parity game) For any formula (j> of modal pt, and a model define a 
parity game ^ x (/> with positions (s, y) where 5 is a state of ^ and i/r is either a proper subformula 
of 0, or the formula \lSS, or the formula ()y for ^^y and i/r s ^ in 0. The initial position is 
(^ 0 ,0) where is the root of Positions ( 5 , i/r) where i/r is a disjunction or ()y' belong to Even while 
conjunctions and positions belong to Odd. Other positions have at most one successor; let them be 
Even’s although the identity of their owner is irrelevant. There are edges from ( 5 , t/rV i/r') and ( 5 , i/r A i/r') 
to both {s,y) and {s,y')', from (s,lJ,X.^) and {s,vX.^) to (i',0); from {s,X) to {s,vX.y) itX is bound 
by V, or ( 5 ,/rX.i/r) if it is bound by /i; finally, from fo every (s',\/ ffS) where {s,s') is an edge in 

./#, and also to every (s, ()y) where Y ^ ^ and from {s,()y) to every (s', y) where (^■,^0 i^ an edge in 
the model Positions {s,P),{s, ->P),{s,T) and ( 5 , _L) have no successors. The parity function assigns 
an even priority to ( 5 , T) and also to {s,P) if s satisfies P in ^ and fo (s, ->P) if s does nof satisfy P in 
otherwise {s,P) and {s, ->P) receive odd priorities, along with (s, _L). Fixpoint variables are given positive 
integer priorities such that v-bound variables receive even priorities while /r-bound variables receive odd 
priorities. Furthermore, whenever X has priority i, Y has priority j and i < j, X must not appear free in 
the formula Y binding Y in jxY.Y or vT.i/r. In other words, inner fixpoints receive lower, less significant 
priorities while outer fixpoint receive high priorities. Other nodes receive the least priority used, 0 or 1. 

We now use parity games to define the semantics of L^. 

Definition 7. (Satisfaction relation) A structure rooted at sq is said to satisfy a formula 'P of L^, 
written ^ |= 'P if and only if the Even player has a winning strategy from (i'o,'P) in ^ x 'P . 

Note that the definition of the model-checking parity game requires a priority assignment to fixpoint 
variables in a formula that satisfies the conditions that v-variables receive even priorities, /i-variables 
receive odd priorities and whenever X has priority i, Y has priority j and i < j, X must not appear free 
in the formula Y binding Y in pY.Y or vT.i/r. For any formula, there are several valid assignments. 
For example, one could assign a distinct priority to every fixpoint, with the highest priority going to the 
outermost bound fixpoint and the priorities decreasing the further into the formula a fixpoint is bound. 
We further restrict a parity assignment to be surjective into an initial fragment of N: if a priority is 
unused, all greater priorities can be reduced by 2. We define fhe alternation depth of a formula to be 
the minimal valid assignment. Although variations of this definition exists, our motivation is to match 
closely the alternations required in the model checking parity game. 

Definition 8. Fet a priority assignment be a function D. : Var {0...n} for some integer n, which is 
surjective on at least such that if fl{X) < D.{Y) then X does not appear free in the formula 

binding Y and the parity of is even for v-bound variables and odd for /r-bound variables. We 
don’t require the priority 0 to be used, but include it in the co-domain for simplicity. In this paper, we 
take the alternation depth of a formula to be the co-domain of the least priority assignment of a formula. 
The correspondance with the priorities of the model checking parity game should make it clear that 
this definition is equivalent to the more typical syntactic ones in the literature, for example in |!2l. An 
alternation free formula is a formula which has both priority assignements with co-domain {0,1} and 
{0,1,2} where 0 is not used. 

Deciding whether a formula is equivalent to a formula with smaller alternation depth is a long stand¬ 
ing open problem. 
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2.2 Tableau decomposition 

Definition 9. (Tableau) A tableau ^ = {T,L) of a formula consists of a potentially infinite tree T of 
which each node n has a label L{n) C sf(^) where sf{^) is the set of proper subformulas of The 
labelling respects the following tableau rules with the restriction that the modal rule is only applied where 
no other rule is applicable. 


{r,i/rA(/)} 


(A) 


{r,^} {r,i/r} 

{r,i/rV(/)} ^ ^ 


{r,aX4} 


(a) with a G {ft, v} 


{r,0} 

{r,x} 


(X) where A is a fixpoint variable bound by aX.^, with a G 


{i/r} U {y G r,^ 7 ^ for every G F, i/r G 

{T} 


Note that each branching node is either a choice node, corresponding to a disjunction, or a modal 
node. Although the rules only contain a binary disjunctive rule, we may write, for the sake conciseness, 
a sequence of binary choice nodes as a single step. Also note that when a modal rule is applied, all 
formulas in a label are either modal formulas or literals, that is to say propositional variables and their 
negations. The latter form the modal node’s set of literal and are a semantically important component of 
the tableau. An inconsistent set of literals is equivalent to ± and a node with such a set of literals in its 
label has no successors. 

Sequences of subformulas along a path in the tableau are called traces and correspond to plays in the 
model checking parity game. A /r-trace is a trace winning for the Odd player. 

Definition 10. (jU.-trace) Given an infinite branch in a tableau, that is to say a sequence noni... of nodes 
starting at the root, where is a child of n,-, a trace on it is an infinite sequence fof\... of formulas 
satisfying the following: each formula is taken from the label of the corresponding node, /,• G L(n,) for 
all i > 0; successive formulas /, and /,+i are identical if /, is not the formula that the tableau rule from 
rii to n,+i acts on; if the tableau rule from n, to is a disjunction, conjunction, or fixpoint binding 
elimination acting on /„ then /i+i is an immediate subformula of fy if the tableau rule from n, to is 
a modality, then /, has to be a formula —and fi^i is either V or a formula i/r G if the tableau 
rule from n,- to is a fixpoint regeneration acting on the fixpoint variable /,-, then is the binding 
formula for /,. A trace is a /r-trace if the most significant fixpoint variable that regenerates infinitely 
often on it is a /r-variable. 

Since labels are to be thought of as conjuncts, it is sufficient for an infinite path in a tableau to allow 
one /r-trace for the infinite path to be winning for the Odd player. 

Definition 11. (Parity of a path) An infinite path in a tableau is said to be even if there are no /r-traces 
on it, otherwise it is said to be odd. 

Note that the order of applications of the tableau rules is non deterministic so a formula may appear 
to have more than one tableau. However, tableau equivalence, defined nexf, only looks af fhe sfrucfure of 
branching, whefher branching nodes are modal or disjuncfive, fhe literals af modal nodes and fhe parify 
of infinite pafhs, so a formula has a unique fableau, up fo fableau equivalence. We define fableau cores 
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to be the semantic elements of the tableau - node types, literals at modal nodes, branching structure and 
the parity of infinite paths - which do not depend on the syntax of the generating formula. Finally, we 
define frees wifh back edges which are finife represenfafions of fableau cores. 

Definition 12. (Tableau core) A fableau core is ^ = (C,fl) where C is a pofenfially infinife buf still 
finifely branching free of which fhe nodes are eifher modal nodes or disjuncfive nodes and modal nodes 
are decorafed wifh a sef of liferals. is a parify assignmenf wifh a finife prefix of N as co-domain. An 
infinife pafh in ^ is of fhe parify of fhe mosf significanf priorify seen infinifely offen. ^ = (C,fl) is a 
fableau core for = (r,L) if once fhe sequences of disjuncfions in are collapsed info one non-binary 
disjunction fhere is a bijecfion b befween fhe branching nodes of T and fhe nodes of C which respecfs 
the following: the successor relation in the sense that b{i) is a child of b{j) in C if and only if i is a child 
of j in T, whether nodes are modal or disjunctive, the literals at modal nodes, and the parity of infinite 
paths. That is to say, if a path in maps to a path in ^ then the highest priority seen infinitely often on 
the path in ^ is even if and only if the path in ^ has no /r-trace. 

Definition 13. (Tableau equivalence) Two tableaus (J^,Lo) and are equivalent if their cores are 

bisimilar with respect to their branching structure, whether nodes are disjunctive or modal, the literals 
at modal nodes and the parity of infinite branches. Two formulas are tableau equivalent if they generate 
equivalent tableaus. 

Definition 14. (Tree with back edges) Tableaus are potentially infinite but regular, so they allow finite 
representations. A finite representation of a tableau = (A,fl) is a finite tree with back edges, A which 
is bisimilar to the core of the tableau. Every node is either a modal node or a disjunctive node and modal 
nodes are associated with a set of literals. The tree has a priority assignment Q. which assigns priorities 
to nodes such that the highest priority on an infinite path is of the parity of that path. 

To summarise, a tableau ^ is a potentially infinite tree labelled with sets of subformulas - it is 
specific fo fhe formula which labels ifs roof; a fableau core, ^ is a pofenfially infinife objecf which 
carries fhe same semantics buf is nof specific fo one formula; finally, a free wifh back edges, called £/ 
because of ifs resemblance fo alfernafing parify aufomafa, is a finife represenfafion of a fableau core. 
The nexf secfion will presenf fhe one-lo-one correspondence befween disjuncfive formulas and frees wifh 
back edges. 

Theorem 15. j[9]/ Tableau equivalent formulas are semantically equivalent. 

Nofe fhaf fableau equivalence is a sfricfer nofion fhan semantic equivalence; t/rV and T have 
differenl fableau for example. 

2.3 Disjunctive normal form 

Disjuncfive form was infroduced in f91 as a synfacfic resfricfion on fhe use of conjuncfions. If forces 
a formula fo follow a simple sfrucfure of alfernafing disjuncfions and modalities where modalifies are 
qualified wifh a conjuncfion of proposifions. Such formulas are in many ways well-behaved and easier 
fo manipulate fhan arbifrary formulas. 

Definition 16. (Disjunctive formulas) The sef of disjuncfive form formulas of (unimodal) is fhe 
smallesf sef salisfying: 

• -L,T, propositional variables and fheir negations are in 

• If y/ € ^ and ^ ^ fhen yr V 0 G ; 

• If is a sef of literals and C ^ (dd is finife), fhen /\£/ A -aSS G ; 
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• nX.Y and vX.y as long as i/a g 

Every formula is known to be equivalent to an effectively computable formula in disjunctive form [Qj. 
The transformation into disjunctive form involves taking the formula’s tableau decomposition and com¬ 
pressing the node labels into a single subformula. The tricky part is finding a tree with back edges and its 
priority assignment to represent the tableau finitely, including the parity of infinite paths. The transfor¬ 
mation then turns the tree with back edges into a disjunctive formula with alternation depth dependent on 
the priority assignment. Conversely, a disjunctive formula and its minimal priority assignment induces a 
tree with back edges representing its tableau. The minimal priority function required to finitely represent 
a tableau is therefore equivalent to the minimal alternation depth of a disjunctive formula generating 
the tableau. The following theorem recalls the construction of disjunctive formulas from trees with back 
edges labelled with priorities from (21 and shows that the alternation depth of the resulting formula stems 
from the priority assignment of the tree with back edges. 

Theorem 17, Let ^ = (A, LI) be a tree with back edges that is bisimilar to a core of the tableau fX 
with priority assignment LI with co-domain {0...^}. Then there is a disjunctive formula with alternation 
depth {0...^} which generates a tableau equivalent to LX. 

Proof First of all, we construct sX' = bisimilar to sX but with a priority assignment with the 

following property: on all paths from root to leaf, the priorities of nodes that are the targets of back edges 
occur in decreasing order. This is straight-forward by looking at the infinite tableau core sX unfolds into, 
remembering which nodes stem from the same node in sX and their priority assigned by LI. First consider 
all branches that see the highest priority q infinitely often and cut them short by creating back edges at 
nodes of priority q, pointing to the bisimilar ancestor node (also of priority q) that is closest to the root. 
Then repeat this for each priority in decreasing order, but for each priority q—\ treat the ancestor of 
priority q that back edges point to (if it exists) as the root, so that nodes that have back edges pointing 
to them end up in decreasing order of priority. Note that every cycle is now dominated by the priority of 
the first node from the root seen infinitely often. 

The disjunctive formula is then obtained by assigning a subformula f{n) to every node of A as 
follows. If n is a leaf with literals Q, then f{n) = /\2; if n is a disjunctive node with children uq and 
ni, then f{n) = f{no) V/(ni); if n is the source of a back edge of which the target is m, then f{n) = X„, 
where X,„ is a fixpoint variable; if n is a modal node, then f{n) = f\Q/\ where Q is the set of literals 
at n and Sd is the set of /(«;) for n, children of n; other nodes inherit the formula assigned to their unique 
child. If n is the target of a back edge, fin) is obtained as previously detailed but in addition, it binds the 
fixpoint variable X„ with a v-binding if n is of even parity and with a /i-binding otherwise. 

If r is the root node of A', then f{r) is a disjunctive formula that generates a tableau that is equivalent 
to .IX. This should be clear from the fact that the tableau of f{n) consists of the infinite tree generated 
by A' and the labelling L{n) = {/(«)} for all n. Ll' restricted to the target of back edges is a priority 
assignment for the disjunctive formula 'T = f{n) since it respects the parity of paths and on each branch 
the priorities occur in decreasing order. This guarantees that if Ll'{X) < LlfY) then X is not free in the 
formula binding Y. 

Therefore 'T has a tableau that is equivalent to fL and accepts a priority assignment with co-domain 

{0...^}. □ 

Conversely, a disjunctive formula induces a tree with back edges generating its tableau by taking its 
tableau until each branch reaches a fixpoint variable which is the source of a back edge to its binding 
formula. The priority assignment of the formula is also a priority assignment for the tree with back-edges. 
This yields a one-to-one correspondence between trees with back edges and disjunctive formulas. 
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3 Tableau equivalence preserves alternation depth for disjunctive 

This section argues that all disjunctive formulas generating the same tableau ^ have the same alternation 
depth. The structures used to identify the alternation depth are similar to ones found in [ 8 ] to compute the 
Rabin-Mostowski index of a parity games and the flowers described in |[T2l to find the Rabin-Mostowski 
index of non-deterministic automata. Here I show that tableau equivalence preserves these structures and 
consequently also the alternation depth of disjunctive formulas. 

Definition [18] describes a witness showing that the priority assignment of a tree with back edges 
£/ = {A,Q.) representing ^ requires at least q priorities. This witness is preserved by bisimulation with 
respect to node type, literals and parity of infinite branches. Since all finite representations of a tableau 
are bisimilar with respect to these criteria, they all have the same maximal witness, indicating the 
least number of priorities can be represented with. 

Informally, the witness of strictness is a series of cycles of alternating parity where each cycle is 
contained within the next. 

Definition 18. (q-witness) A (^-witness in a tree with back edges (A,D) representing a tableau consists 
of q cycles ci ...c^ such that for each i < q, the cycle c, is of the parity of i and for all 0 < i < q, the cycle 
c, is a subcycle of c/+i. 

Lemma 19. If a tree with back edges (A,fl) has a q-witness, then the co-domain of the priority assign¬ 
ment Q has at least q elements. 

Proof Given a ^-witness ci ...c^, for every pair of cycles c,- and c,+i, since they are of different parity and 
c, is contained in c,+i, the dominant priority on c,+i must be strictly larger than the dominant priority on 
c,. Therefore there must be at least q priorities in the cycle Cq which contains all the other cycles of the 
witness. □ 

Lemma 20. If a tree with back edges sf representing a tableu does not have a q-witness, then there is 
an tree with back edges sA' which also represents SA but has a priority assignment with fewer priorities. 

Proof. Assume a tree with back edges sA = (A,fl) representing with a priority assignment with co¬ 
domain { 0 ...( 7 } does not have a q witness. Let Sq be the set of nodes of priority q. Let S,_i for 1 < / < <7 
be the set of nodes of priority i — 1 which appear as the second highest priority in a cycle where all the 
nodes of highest priority are in Si, and as the nodes of highest priority in some cycle. Note that if 
was non-empty, then there would be a ^-witness, so and consequently Sq must be empty. Then define 
a new priority function as follows: the new priority function fl' is as fl, except for nodes in any S, - 
these receive the priority i — 2 instead of the priority i. Since Si and Sq are empty, this is possible whilst 
keeping all priorities positive. D' with co-domain { 0 ...^— 1 } preserves the parity of infinite branches 
since there are no cycles in which the priority of all dominant nodes is decreased more than the priority 
of all sub-dominant nodes and each node retains the same parity. Therefore, if a finite representation of 
.If" does not have a ^-witness, then there is a finite representation sA' = (A,D') with a smaller priority 
assignment. □ 

Lemma 21. All tableau equivalent trees with back edges have the same q-witnesses: for all q, either all 
or none of the trees with back edges representing a same tableau fA have a q-witness. 

Proof. First we recall that if is the finite representation of fA induced by a disjunctive formula 
then the tableau of fA is an infinite tree bisimilar to with respect to node type, literals and parity of 
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infinite branches. Hence any finite representation of ^ is bisimilar to £/. It then suffices to show that 
^-witnesses are preserved under bisimulation. This is straight-forward: let £/' be bisimilar to a finite 
tree with back edges £/ with respect to node type, literals at modal nodes and the parity of infinite paths. 
Then infinite paths in £/ are bisimilar to infinite paths in £/'. Since both £/ and £/' are finite, an infinite 
path stemming from a cycle in is bisimilar to a cycle in . A ^-witness contains at least one node 
which lies on all the cycles of the witness. If ^ has q cycles, call the node on all of its cycles n and 
consider (one of) the deepest node(s) n' in bisimilar to n. That is to say, choose n' such that if another 
node bisimilar to n' is reachable from n', it must be an ancestor of n'. Since n' is bisimilar to n, there 
must be a cycle c- bisimilar to each ci reachable from n'. Since n' is maximally deep, it is contained in 
each of these cycles c-. Then, a ^-witness can be reconstructed in by taking the cycle Cj, and then 
for each i > 0 the cycle consisting of all c'jj<i . Since all c- cycles have n' in common, there is a cycle 
combining c':,j < i for any i. Since bisimulation respects the parity of cycles, this yields a < 7 -witness in 

□ 


Theorem 22. All disjunctive formulas with tableau fA have the same alternation depth. 

Proof. All trees with back edges representing the same tableau ^ have the same maximal witness, 
from the previous lemma, so from Lemma |20] they accept a minimal priority function with domain 
{0...^}. Since a disjunctive formula induces a tree with back edges with a minimal priority function 
corresponding to the formula’s alternation depth, any two disjunctive formulas that are tableau equivalent 
must have the same alternation depth. □ 

This concludes the proof that tableau equivalence preserves alternation depth on disjunctive formulas. 
The restriction to disjunctive formulas is crucial: as the next section shows, in the general case tableau 
equivalent formulas may have vastly different alternation depths. 

4 Disjunctive form does not preserve alternation depth 

Every formula has a tableau which allows it to be turned into a semantically equivalent disjunctive 
formula. This section studies the relationship between a formula’s alternation depth and the alternation 
depth of its tableau equivalent disjunctive form. As the previous section shows, any two disjunctive 
formulas with the same tableau have the same alternation depth; therefore comparing a non-disjunctive 
formula to any tableau equivalent disjunctive formula will do. 

The first subsection demonstrates that not only does disjunctive form not preserve alternation depth, 
but also that there is no hope for bounding the alternation depth of disjunctive formulas with respect to 
their semantic alternation depth: for any n there are one alternation formulas which are tableau equiva¬ 
lent to n alternation disjunctive formulas. In other words, the alternation depth of a formula, when 
transformed into disjunctive form, can be arbitrarily large. Conversely, as shown in the second subsec¬ 
tion, formulas of arbitrarily large alternation depth can be tableau equivalent to a disjunctive formula 
without alternations. Hence the alternation depth of tableau equivalent formulas are only related within 
the disjunctive fragment. 

4.1 Disjunctive formulas with large alternation depth 

While the main theorem is proved by Example [27l the Examples |23] and [25] leading up to it should give 
the interested reader some intuition about the mechanics which lead the tableau of a formula to have 
higher alternation depth than one might expect. 
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Y,X W 


>{y},AA^{X} AA^{W} 


Y Z 


A,—>-{Z} 
^{F},A AA^{Z} 


* ^{F},(AA^{A})VA (AA^{W})V(AA^{Z}) 

VF.->{F} A/xX.(AA->{X}) VA vZ./xW.(A A ^{W}) V (A A-^{Z}) 


Figure 4.1: Tableaus for vF.—)-{F} A lxX.(AA^{X}) VA and vZ./rW.(A A ^>{W}) V (A A—)-{Z}) 


Example 23. The first example is a rather simple one: a disjunctive formula with one alternation that 
can be expressed in non-disjunctive form without any alternations. The disjunctive formula vX./iF.(A A 
— 5 -{X}) V (A A —>^{F}) signifies that all paths are infinite and A occurs infinitely often on all paths. Com¬ 
pare it to the formula vA.—)-{A} A /iF.(A A ^{F }) V A which is alternation free. 

The tableaus of both these formulas are shown side by side in Figure 14.11 Both branches regenerate 
into either exactly the ancestral node marked * * or a node that reaches a node identical to the one marked 

* in a single non branching step. 

The cores of the two tableaus, that is to say their branching nodes, are clearly isomorphic with respect to 
the node type and branching structure. Furthermore, for both formulas, there is /X-trace on any path that 
only goes through the left hand branch infinitely often. There is no jx trace on any path that goes through 
the right hand path infinitely often, for either formula. As a result, both tableaus agree on the parity of 
infinite branches. The two formulas are tableau equivalent and therefore also semantically equivalent. 

Remark 24. Observe that there is nothing obviously inefficient about how the disjunctive formula han¬ 
dles alternations. Indeed, simply inverting the order of the fixpoints yields a formulas which can not be 
expressed without an alternation: jxX.vYA A —} VA A —>{F}. 

While the above example proves that disjunctive form does not preserve alternation, it must be noted 
that the alternating parity automata corresponding to these formulas require in both cases two priorities, 
although only one requires an alternation. The next example shows formulas in which the number of 
priorities is not preserved either. 

Example 25. This example and the following ones will be built on one-altemation formulas consisting 
of single pt/v alternations embedded in one another without interfering with each other, i.e. all free 
variables within the inner formula (pi are bound by the inner fixpoint bindings. This means that the 
formula accepts a priority assignment with co-domain {0,1}. Without further ado, consider the formula 
in question: 

a = lxXo.vYo.{A A ^{Ao}) V (B A ^{Fo) A lxXi.vYi.{C A ^{Ai}) V (D A ^{Fi}) V E 

The following Lemma shows it to be equivalent to a formula which requires a priority assignment 
with co-domain {0...3}. 

Lemma 26. The formula a is tableau equivalent to a disjunctive formula which requires a parity as¬ 
signment with co-domain {0...3}.' 


p = IxXo.vYo.lxXi.vYifA A C A ^{Aq}) V (A AD A ^{Aq}) V (A AE A ^{Aq}) 
V(EAEA^{Fo}) V(EACA^{Ai}) V(EADA^{Fi}) 


(4.1) 
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* 

* 

* 

* 

* 

* 

Yq,Yi 

Yo,Xl 

Yo 

Xo,F, 

Xo,Xi 

Xo 


{B,^{yo},c,->{x,}) 

{B,^{y„},£) 

(A,^{X„},D,^{y,}) 

{A,->-{Xo},C,—i{Xi }) 

(A,^{Xo},£) 

(BA^{yo}),{DA^{y,}) 

(BA^{yo}),(CA^{X,}) 

(BA^{yo},£) 

(AA^{Xo}),(DA^{y,}) 

{AA^{Xo}),(CA^{y,}) 

{AA^{Xo},£) 

(BA^{Fo}),(CA^{X|})V(DA^ 

{Fi})V£ 

(AA^{Xo}),(CA^{Xi})V(DA^{Fi})V£ 


*(A A ^{Xo}) V (B A ^{Fo}), (C A }) V (D A ^ 

{FiDVB 



/iXo.vFo.(AA^{Xo})V(BA^{Fo})A/iXivFi(CA^{Xi})V(DA^{F|})V£ 




Figure 4.2: 

Tableau for a 



* 

* 

* 

* 

* 

* 

F 

Xi 

lo 

Xo 

Xo 

Xo 

{B.A^{F,}) 

{B.C.^iXi}) 


(A.D.^IXoj) 

(A,C,^{Xo}) 

{A,£,^{Xo}) 

(BADA->{Fi}) 

(BACA^{Xi}) 

{BAEA^lYo}) 

(AADA^{Xo}) 

(AACA^{Xo}) 

(AA£A^{Xo}) 

(BA£A->{Fo}) V{BACA^{Xi})V(BADA^{Fi}) 

(AA£A^{Xo})V(AA£)A^{Xo})V(AACA->{Xo}) 


* (AA£A^{Zo}) V(AAZ)A^{Xo}) V(AACA^{Zo})V(BA£A^{yo}) V(BACA^{Zi}) V(BADA^{F|}) 


AiZo.vFo./jXi.vFi.(AA£A^{Zo})V(AADA^{Zo})V(AACA-s>{Zo})V(BA£A^{yo})V(BACA^{Zi}) V{BADA->{yi}) 

Figure 4.3: Tableau for (5 


Proof. The tableaus for both formulas are written out in Figures l4~2] and 1431 The two tableaus are iso- 
morphie with respeet to branching structure, node type and the literals at modal nodes. To prove their 
equivalence, it is therefore sufficient to argue that this isomorphism also preserves the parity of infinite 
branches, that is to say that there is a /i-trace in an infinite path of one if and only if there is a /i-trace in 
the corresponding infinite path of the other. 

To do so, we look, case by case, at the combinations of branches that a path can see infinitely often 
and check which have a trace in each tableau. First argue that the three right-most branches in both 
tableaus are such that any path that sees them infinitely often has a /r-trace. This is witnessed in both 
cases by the least fixpoint variable Xq which will dominate any trace it appears on and appears on a trace 
on all paths going through one of these branches infinitely often. So, in both tableaus, any path going 
through one of the right-most branches infinitely often is of odd parity. Now consider the branch that 
ends in Fq before regenerating to the node marked * in both tableaus. All traces on paths that go infinitely 
often through this branch will see Fq regenerate infinitely often. Therefore in both tableaus, a path going 
through this branch infinitely has a jj. trace if and only if it also goes through one of the three rightmost 
branches infinitely often. Now consider the fifth branch from the right, the branch that regenerates Fo,Xi 
in one case and just Xi in the other. In both tableaus, a path that goes through this branch infinitely often 
will have a /r trace unless it goes through the Fq branch infinitely often and doesn’t go through one of the 
three right-most branches infinitely often. Finally, in both tableaus, a branch that only sees the left-most 
branch infinitely often is of even parity since such a path does not admit any /r-traces. However, if a 
path sees this branch and some other branches infinitely often, its parity is determined by one of the 
previously analysed cases. Since we have analysed all the infinite paths on these tableaus and concluded 
that in each case the parity of a path is the same in both tableaus, this concludes the proof that the two 
tableaus are equivalent. □ 

The above example yields a disjunctive formula of alternation depth {0...3} which semantically only 
requires alternation depth {0,1}. This proves that disjunctive form does not preserve the number of 
priorities the model checking game of a formula requires. 
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The next step is to generalise the construction of Example |25] to arbitrarily many alternations to 
prove that there is no bound on the number of alternations of a disjunctive formula tableau equivalent to 
a non-disjunctive formula of n alternations. To do so, we will first define the one-altemation formulas 

inductively, based on the formula of Example |25] We then argue that the tableau of admits a 
{2n + l)-witness, proving that a„ is not tableau equivalent to any disjunctive formula of less than 2n+l 
alternations. Due to the argument pertaining to traces in increasingly large tableaus, its details are, 
inevitably, quite involved. However, the mechanics of the tableaus of a„ are not difficult; writing down 
the tableau of a 2 and working out its disjunctive form should suffice to gain an intuition of the proof to 
follow. 

Example 27. In order to define for any n define: 

ai = AtXi.vTi.((Ai A ^{Xi}) V (Bi A^{Fi}) VEi)A 

AtXo.vFo.(Ao A ^{Xo}) V (Bo A ^{Fq}) VEq (4.2) 

‘^r+i = AtX,-+i.v}j'+i -((^r+i A —^-IXi’+i}) V (B,+ i A —^-iFj+i}) VEi+i) Aa, 

Then, define: 

Otn = lxX„.vY„.{(A„ A —)'{X„}) V A —>-{F„})) Aa„_i 

In other words, the formula consists of nested clauses /rX,'.v}j'.((A,- A —>^{X,}) V (B,- A —)-{F,}) VB,) con¬ 
nected by conjunctions where the outmost clause does not have a VB. 

As the formula grows, its tableau becomes unwieldy, but its structure remains constant: it is just as 
the tableau of a with more branches. Figure IA2] can be used as reference. 

The tableau of any a„ follows this structure: 

• The first choice node {(A„ A {X„ }VB„A^>{F„}),...,(AoA—> {Xq } V Bq A —{Fq } V Bq )} branches 

into 2x3” modal nodes - ignoring the modalities attached to each literals for a moment, this is the 
decomposition of (A„ VB„) A (A„_i VB„_i VB„_i)... A (Aq VBq VBq) into one large disjunction. 

• Each choice leads to a modal node with some choice of propositional variables consisting of one 
of An and B„ and then for every i < n one of A, ,B, or B, . 

• These modal nodes have a single successor each, consisting of a set of fixpoint variables. In every 
case, one of these is F„ or X„ and there is only ever at most one fixpoint variable out of {X, , Ij } for 
each i. These nodes will be referred to as regeneration nodes. When a regeneration node does not 
contain X, nor F, for some /, this corresponds to B, having been chosen rather than A, or B,. 

• Nodes consisting of a set of fixpoint variables all regenerate, give or take a couple of non-branching 
steps, into the same choice node, identical to the ancestral choice node labelled: 

{{An A —T'jX^} VB„ A ^{F„}),..., (Aq A ^{Xq} VBq A —^{Fo} V Bq)} 

• An infinite trace in this tableau sees infinitely often only fixpoint variables Yj and/or X, for some 
i. As a consequence if a path goes infinitely often through a regeneration node which does not 
contain X; or Ij-, then there is no trace that sees X, infinitely often on that path. 

Lemma 28. The formula a„ is tableau equivalent only to disjunctive formulas which require a priority 
assignment with 2n -|- 1 priorities. 
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Proof. Using the above observations, we will show that the tableau for this formula requires at least 
2n + 1 alternating fixpoints. We describe a priority assignment to a subset of the nodes of the tableau of 
a„ such that on the paths within this subset, a path is even if and only if the most significant priority seen 
infinitely often is even. We then argue that this subset constitutes a 2?i + 1-witness. 

Consider the paths of the tableau which only contain the following regeneration nodes: 

• For all i, the nodes regenerating exactly and 

• For all i the nodes regenerating exactly F„...F,+i2f,F,_i...Fo. 

For each i, assign priority 2i to the node regenerating F„...F; and 2/-|- 1 to the node regenerating 
F„...U+i ,X/,F;_i,...Fq. We now prove that this priority assignment is such that a path within this sub¬ 
tableau is even if and only if the highest priority seen infinitely often is even. 

First consider the nodes Y„...Yi, which have been assigned even priority. A path that sees such a node 
infinitely often can only have a -trace if it sees a node regerating some Xj, j > i infinitely often. Such 
a node would have an odd priority greater than Yn-.-Yf. Therefore, if the most significant priority seen 
infinitely often is even, the path has no jj. trace. Conversely, if a path sees F„...A,...Fo infinitely often and 
no Y„...Yj where j > i infinitely often, then there is a trace which only regenerated A, and U infinitely 
often. This is a trace since X, is more significant than F,. This priority assignment therefore describes 
the parity of infinite paths on this subset of paths of 

Any assignment of priorities onto ^ should, on this subset of paths, agree in parity with the above 
priority assignment. However, in any tree with back edges generating this tableau, this subset of paths 
constitutes a 2n-|- 1 witness: cq is a cycle that only sees F„...Fo, ci contains cq and also sees F„...AiFo 
infinitely often and for all i > 1, the cycle C2i is one containing C2i-i and Y„...Yi while C2/+1 is one 
containing C2i and F„...A,-...Fo. Each cycle c/ is dominated by the priority j, making cq, ...,C 2 i+i a 2i 1- 
witness. Thus, using Theorem [22] any disjunctive formula with tableau ^ must require at least 2n -|- 1 
priorities. □ 

This concludes the proof that for arbitrary n, there are one-alternation formulas which are tableau 
equivalent to disjunctive formulas with n alternations. 

4.2 Disjunctive formulas with small alternation depth 

The previous section showed that transforming a formula into disjunctive form can increase its alternation 
depth. The converse is much easier to show: there are very simple formulas for which the transformation 
into disjunctive form eliminates all alternations. 

Lemma 29. For any formula y> the formula (/rA.—>-{A} V —>^-L) A t/r is tableau equivalent to a disjunc¬ 
tive formula without V-operators. 

Proof. The semantics of (/rA.—)-{A } V — 7>-L) A t/r are that a structure must not have infinite paths and y/ 
must hold. Consider IX , the tableau for (/lA.—)-{A} V —>--L) A t/r. It is easy to see that every modal node 
will either contain —>-{A} or —The latter case terminates that branch of the tableau, while the former 
will populate every successor node with A which will then regenerate into (—)-{A} V —>^-L). As a result, 
all infinite paths have a jj. trace; there are no even infinite paths. Any disjunctive formula generating 
will therefore only require the jJ. operator. □ 

Taking i/r to be a formula of arbitrarily high alternation depth, (/rA.—^{A} V ^-L) A y/ shows that 
the transformation into disjunctive form can reduce the alternation depth an arbitrarily large amount. 
Together with the previous section, this concludes the argument that there are no bounds on the difference 
in alternation depth of tableau equivalent formulas. 
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5 Discussion 

To summarise, we have studied how tableau decomposition and the transformation into disjunctive form 
affects the alternation depth of a formula. The first observation is that within the confines of the dis¬ 
junctive fragment of L^, alternation depth is very well-behaved with respect to tableau equivalence: any 
two tableau equivalent disjunctive formulas have the same alternation depth. However, the story is quite 
different for without the restriction to disjunctive form: the alternation depth of a formula can not 
be used to predict any bounds on the alternation depth of tableau equivalent disjunctive formulas and 
vice versa. 

Part of the significance of this result are the implications for our understanding of the alternation 
hierarchy. 

The formulas in Section 4 illutrate some of the different types of accidental complexity which 
any procedure for deciding the alternation hierarchy would need to somehow overcome. The formula 
(/xX.—>{X} V _L) A t/r, from Lemma |29] which is semantically a v-free formula for any y is an example 
of a type of accidental complexity which the tableau decomposition eliminates. However, the formula in 
Example |23] illustrate a more subtle form of accidental complexity that is immune to disjunctive form: 
vX./iF.(A A —>'{X}) V (A A —)-{F}) is semantically alternation free while the syntactically almost identi¬ 
cal formula /rX.vF.(A A —>^{X}) V (A A —)-{F}) is not. These formulas pinpoint a very specific challenge 
facing algorithms that try to reduce the alternation depth of formulas; as such, they are valuable case 
studies for those seeking to understand the alternation hierarchy. 

Finally, we showed that the following is decidable: for any formula, the least alternation depth 
of a tableau equivalent disjunctive formula is decidable. This raises the question of whether the same is 
true if we lift the restriction to disjunctive form, but keep the restriction to tableau equivalence: for a 
formula, is the least alternation depth of any tableau equivalent formula decidable? Tableau equivalence 
is a stricter equivalence to semantic equivalence, so this problem is likely to be easier than deciding 
the alternation hierarchy with respect to semantic equivalence but it would still be a considerable step 
towards understanding accidental complexity in L^. 

Acknowledgements I thank the anonymous reviewers for their thoughtful comments, which have 
helped improve the presentation of this paper and relate this work to similar results for other automata. 
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